The following references are helpful to understand the present disclosure and are hereby incorporated herein by reference:                [1] Y. S. Ong, P. B. Nair, and A. J. Keane, “Evolutionary optimization of computationally expensive problems via surrogate modeling,” AIAA Journal, vol. 41, No. 4, 2003, pp. 687-696.        [2] K. Hamza and K. Saitou, “Vehicle crashworthiness design via a surrogate model ensemble and a co-evolutionary genetic algorithm,” Proc. of ASME International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, Long Beach, Calif., September 2005.        [3] S. Obayashi, D. Sasaki, Y. Takeguchi, and N. Hirose, “Multiobjective evolutionary computation for supersonic wing-shape optimization,” IEEE Transactions on Evolutionary Computation, vol. 4, No. 2, 2000, pp. 182-187.        [4] Z. Zhou, Y. S. Ong, M. H. Nguyen, D. Lim, “A study on polynomial regression and Gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm,” Proc. of IEEE Congress on Evolutionary Computation, Edinburgh, United Kingdom, September 2005.        [5] S. Dutta, D. Misra, R. Ganguli, B. Samanta and S. Bandopadhyay, “A hybrid ensemble model of Kriging and neural networks for ore grade estimation,” International Journal of Surface Mining, Reclamation and Environment, vol. 20, no. 1, 2006, pp. 33-45.        [6] J. M. Twomey and A. E. Smith, “Committee networks by resampling,” in Intelligent Engineering Systems through Artificial Neural Networks, C. H. Dagli, M. Akay, C. L. P. Chen, B. R. Fernandez and J. Ghosh, Eds. ASME Press, 1995, vol. 5, pp. 153-158.        [7] A. Krogh, J. Vedelsby, “Neural network ensembles, cross validation, and active learning,” in Advances in Neural Information Processing System 7, Cambridge, Mass.: MIT Press, 1995, pp. 231-238.        [8] G. Brown, J. Wyatt, R. Harris and X. Yao, “Diversity creation methods: A survey and categorization,” Journal of Information Fusion, vol. 6, no. 1, January 2005, pp. 5-20.        [9] Y. Liu, X. Yao, “Ensemble learning via negative correlation,” Neural Networks, vol. 12, pp. 1399-1404.        [10] M. Islam, X. Yao, “A constructive algorithm for training cooperative neural network ensembles,” vol. 14, no. 4, pp. 820-834.        [11] G. P. Coelho and F. J. Von Zuben, “The influence of the pool of candidates on the performance of selection and combination techniques in ensembles,” in Proc. of the International Joint Conference on Neural Networks, Vancouver, BC, Canada, 2006, pp. 10588-10595.        [12] J. Torres-Sospedra, M. Femandez-Redondo, and C. Hernandez-Espinosa, “A research on combination methods for ensembles of multilayer feedforward,” Proc. of International Joint Conference on Neural Networks, 2005, pp. 1125-1130        [13] D. Chen, J. A. Quirein, H. D. Smith, S. Hamid, J. Grable, “Neural network ensemble selection using a multi-objective genetic algorithm in processing pulsed neutron data,” Society of Petrophysicists Well Log Analysts (SPLWA)45th Annual Logging Symposium, Jun. 6-9, 2004, Noordwijk, The Netherlands.        [14] D. Chen, J. A. Quirein, H. Smith, S. Hamid, J. Grable, and S. Reed, “Variable input neural network ensembles in generating synthetic well logs,” Proc. of International Joint Conference on Neural Networks, Vancouver, BC, Canada, 2006, pp. 2273-2280.        [15] Y. Jin, T. Okabe, and B. Sendhoff, “Neural network regularization and ensembling using multi-objective evolutionary algorithms,” in Proc. Congress on Evolutionary Computation, Portland, Oreg., 2004, pp. 1-8.        [16] H. A. Abbass, “Pareto neuro-evolution: Constructing ensemble of neural networks using multi-objective optimization,” in Proc. Congress on Evolutionary Computation, Can berra, Australia, 2003, pp. 2074-2080.        [17] A. Chandra and X. Yao, “DIVACE: Diverse and accurate ensemble learning algorithm,” in The Fifth International Conference on Intelligent Data Engineering and Automated Learning, Exeter, UK, 2004, pp. 619-625.        [18] P. Castillo, M. Arenas, J. Merelo, V. Rivas, and G. Romero, “Multiobjective optimization of ensembles of multilayer perceptrons for pattern classification,” in Parallel Problem Solving from Nature IX, Reykjavik, Iceland, 2006, pp. 453-462.        [19] Y. Jin, M. Olhofer, and B. Sendhoff, “A framework for evolutionary optimization with approximate fitness functions,” IEEE Transactions on Evolutionary Computation, vol. 6, no. 5, 2002, pp. 481-494.        [20] B. S. Yang, Y. Yeun, and W. Ruy, “Managing approximation models in multiobjective optimization,” in Structure and Multidisciplinary Optimization, vol. 24, no. 2, 2002, pp. 141-156.        [21] R. Maclin, J. W. Shavlik, “Combining the predictions of multiple classifiers: using competitive learning to initialize neural networks,” in Proc. of the 14th International Joint Conference on Artificial Intelligence, Montreal, Canada, 1995, pp. 524-530.        [22] P. Sollich and A. Krogh, “Learning with ensembles: how over-fitting can be useful,” in Advances in Neural Information Processing Systems 8, D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, Eds. Cambridge, Mass.: MIT Press, 1996, pp. 190-196        [23] R. S. Renner, “Combining constructive neural networks for ensemble classification,” in Proc. Of the International Joint Conference on Intelligent System, Atlantic City, N.J., 2000, pp. 887-891.        
Usage of high-fidelity simulation tools such as Finite Element Analysis (FEA) and Computational Fluid Dynamics (CFD), for example, has become standard practice in engineering today. However, the expensive computational cost associated with running such simulation tools is often prohibitive, preventing engineers from conducting enough simulations to discern an optimal design. To address this issue and facilitate product optimization, engineers have in some cases developed surrogate models that are computationally efficient, robust, and can be used for preliminary analysis before unleashing the high-fidelity simulation tools on selected designs. The surrogate models can be incorporated into a search engine to locate potentially feasible designs and to identify design problem areas [1-3].
Several surrogate modeling techniques (neural networks, polynomial regression, Gaussian process, etc.) are available today. The most suitable surrogate model technique will vary based on the specific problem and the engineer's experience [4-5], and the performance of the various techniques can be expected to vary significantly when only a limited amount of design data is available from which to develop the surrogate model. In neural network modeling, for example, an over-trained neural network developed under sparse data conditions will memorize the training data and fail to generalize well on the unseen new data. However, an under-trained neural network whose development is terminated by conventional early-stopping will perform poorly even on the given training examples. Traditionally, the prediction error of a neural network generated from sparse data has been estimated using resampling based cross-validation (leave-one-out) and bootstrap methods [6]. When only a single neural network is employed, the estimated prediction error is usually quite high.
Compared to single neural networks, neural network ensembles offer a more robust surrogate model by combining multiple predictions from diverse member networks. Many studies in this area are related to incorporative training (ambiguity decomposition [7-8], negative correlation learning [9-10]) and selection/combination methods [11-12], but less attention has been paid to surrogate model development from sparse data.
While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the present invention as defined by the appended claims.